Skip to content

Kernelize's Deepseek v4.1 Bringup - #65964

Draft
abrown wants to merge 257 commits into
tenstorrent:mainfrom
abrown:kernelize-bringup-ds41
Draft

abrown wants to merge 257 commits into
tenstorrent:mainfrom
abrown:kernelize-bringup-ds41

Conversation

@abrown

@abrown abrown commented Nov 16, 2026 •

Copy link
Copy Markdown

PR Category

Feature: bring-up of Deepseek v4.1 on a single Galaxy.

Summary

This is a Kernelize project to upstream Tenstorrent support for Deepseek v4.1. It builds on work by @mvasilijevicTT but reworks significant portions to substantially improve prefill TTFT. It has accompanying PRs in other repositories: tenstorrent/vllm-tt-plugin#340, tenstorrent/tt-inference-server#5586.

Notes for reviewers

@bgoelTT, as requested, this branches off of v0.76.0; this means it is considerably out of date with main. Also, it adds 156 commits above those used from @mvasilijevicTT and not all of those need to go in this PR. There are all kinds of bug fixes, debugging helpers, tests, etc., that could become separate PRs for easier review. Let me know how you want to proceed.

Verification

  • verified accuracy with GPQA Diamond, MMLU-PRo, LongBench v2, Terminal Bench 2.1, Tau3 Banking
  • verified performance with AIPerf sweep over 1K-1M input tokens at concurrency 1 and 32

Code owners

  • Ping pending code owners

mvasilijevicTT and others added 30 commits September 28, 2026 23:42
tt/v41: TtV41Attention (q/kv stems without q_b_norm, V4.1 rope tables,
sparse_sdpa over window + compressed KV with per-head sink and the ttMLA
head-to-sequence reshard for 16 heads/chip, grouped O) and TtV41Block
(lagged mHC, TtMoe with the V4.1 config). The compressed-KV path, window-KV
FP8 QDQ and index-row assembly are explicit host fallbacks on the vendored
reference modules. The test runs a 5-layer real-dims schedule standing for
V4.1 layers 2,3,20,21,24 against the CPU oracle on LoudBox 2x4: deterministic;
attention 0.988-0.994, MoE 0.996, block 0.997-0.998 (layer 0: 0.938, shown
to be reference sensitivity to attention error at the first layer).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TtV41Attention takes an optional compute kernel config; the experiment runs
the prototype attention chain for weight-dtype x fidelity variants. Result:
bfp8/bf16 weights and HiFi2/HiFi4 differ by <=0.0005 PCC; the gap to the
reference is its FP8 activation quantize-dequantize (dev-spec D-I).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TtV41HyperConnections applies a block's two mHC sites around caller-given
sublayers with V4.1's lagged pre (attention uses the incoming pre, the FFN the
one split before attention, the block returns the one split before its FFN),
plus the post-model collapse with the last pre (no head mix). TtV41Block uses
it as its only residual path. Against the reference at C=5120 on LoudBox 2x4,
two chained blocks + final collapse reach >= 0.99999 PCC with synthetic and
real layer 2/3 weights, bit-identical on repeat.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One replicated natural-order row-major tensor per KV source:
[128-row window slot | chunk window rows | compressed rows], so a layer's
window KV and its source's compressed KV share one sparse_sdpa index space
without copying the compressed rows. Chunks append only complete ratio groups
of their valid length (padded tail writes nothing); per-SP-rank rows are
gathered and written with slice_write. Per-layer 128-row window carries
persist across chunks. Geometry goldens and a device test over two chunks and
a padded tail pass on LoudBox 2x4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Real-weight blocks need the raw HF shards turned into the exact tensors the
V4.1 device modules consume. weights.py is the host-only preparation phase:
it reads tensors by name with one open per shard, dequantizes FP8 E4M3 32x32
blocks and MXFP4 routed experts (low nibble = even element, E8M0 per 32 along
the input dim) to bf16 exactly, validates stored dtypes and shapes against the
config, and returns the TtV41Block weights dict plus the role-specific
compressor/indexer tensors and gate bias_vl. The checkpoint is pinned to HF
revision dba1be0a by the git blob id of its index; $TT_V41_CHECKPOINT
overrides the HF cache snapshot path.

Device materialization and tensor caches stay with the modules that own the
placement; the device test only proves real layer-2 attention weights place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ratio 1 is norm(wkv(x)) with the bf16 projection; ratio 2 pools each
non-overlapping token pair with a per-channel softmax over fp32 wkv/wgate
projections, then norm. A trailing incomplete pair yields no row and its
fp32 projections are returned as the carry. Shared TP collectives move to
tt/v41/ccl.py (attention uses them too). Against the reference Compressor at
real dims on LoudBox 2x4: >= 0.99998 PCC for synthetic and real layer 2/20
weights, incl. odd length, carry and two chunks vs single shot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…chunk contract (§6)

Block, cache and chunked device tests need trusted expected results for any
subset of V4.1 layers, synthetic or checkpoint weights, without re-running
the minutes-long CPU reference each time. oracle.py builds the vendored
reference for a layer subset at real dims (roles, SWA-only ratio 0, Engram
hash aligned to the full layout) or at small dims, captures per-block
streams/sublayer I/O, the shared state each source publishes and the final
state (window rings, ratio-2 carry, Engram tokens, DSpark rings) with
forward hooks only, and caches results keyed by args, layer ids, seed,
weights revision, tokenizer, prompt hash, reference-source digest, torch
version and PACKAGE_VERSION. chunk_expectations slices the single-shot
result into what a chunked (padded-tail) device run must match
(graph.md §4); the reference itself is only ever run with start_pos 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reference quantize-dequantizes window KV and DSpark rings (FP8 e4m3,
group 32, ue8m0), index keys and indexer queries (FP4 E2M1, group 32,
ue8m0) and compressed KV (FP4 E2M1, group 16, e4m3 scale), and D-I proposes
emulating its FP8 activation QDQ before the quantized GEMMs. Cache and
score bars require the device to reproduce these values exactly given the
same bf16 input, so this adds ttnn-composite ops (dev-spec D-H, F6.1a):
exact group amax, power-of-two scales from the amax bit pattern (checked
against fast_round_scale for every positive bf16), e4m3 RNE via exponent
quantum + floor with exact tie resolution, and E2M1 rounding by exact
threshold comparisons so the non-power-of-two e4m3 scale needs no exact
division. Ops are row-local, so sharded or replicated mesh inputs work.

Tests on the 2x4 mesh compare against kernel_cpu with torch.equal over
random groups across ~40 decades, zero/floor groups, every e4m3/E2M1
half-way point and scale boundary (+-1 bf16 ulp), scale ties and
saturation, W=512/128, S=2048, plus hand goldens and determinism. bf16
subnormal inputs (only relevant for FP4-ue8m0 groups with amax < ~2^-121)
are documented as not reproduced: the device flushes subnormals.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Queries are split over SP then TP, so each chip scores its contiguous
queries with all 32 heads: no cross-chip score reduction, and the selection
lands in the attention's head->sequence query layout. Scores use
indexer_score_dsa with the V4.1 visibility rule applied explicitly (the
kernel's causal offset is not per-chip under this split); candidate blocks
and top-k use topk_large_indices with sentinel tails. qr gets the reference's
FP8 activation QDQ before the exact wq_b; q and keys the FP4 QDQ. At real dims
on LoudBox 2x4: index scores 0.9997, keys 0.9992, selection exact on tie-free
scores for all three index roles, deterministic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scaling FP8/FP4 operands by their (power-of-two or exact) block scales before
a single fp32 matmul is exact per element; only the fp32 summation order
differs from the per-block loop, within the port's documented tolerance.
The per-block Python loop made a 5-layer real-dims oracle exceed 90 minutes on
a loaded host. CPU reference/oracle tests: 28/28 pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The vision tower (graph V1-V3) must turn one image's patches into the aligner rows the image span
merge (10.2) consumes. TtV41Vision replicates the ~0.9 GB encoder on every device and runs it without
collectives, so the replicated [T, 5120] output can be sliced by any SP/TP layout downstream.

Semantics follow inference/vision.py exactly: rotate-half 2D RoPE via rotary_embedding_hf with the
[h*f, w*f] angle vector duplicated over both halves; per-image bidirectional SDPA with
cu_window_seqlens=[0, N, S] so tile/chunk padding never enters attention; the aligner's channel-major
3x3 unfold is replaced by a row-major reshape/pad/kh-slice gather plus a host permutation of w1's
input columns (exact). Qwen-VL classes are not reused because they are bound to tt_transformers
ModelArgs; their windowed SDPA and QKV head split are reused as mechanisms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, 384)

The V4.1 gate matmul (640 tokens/chip, 5120/4 hidden, 384 experts) missed
the tuned table and fell back to TTNN's default tiling, which measures
282 us per device (trace replay, BH) against 15.3 us for the swept 2D
config (6x10 grid, per_core 2x2, in0_block_w 8, subblock 1x2); outputs
are bit-identical. Only the V4.1 key is added, so no existing model's
lookup changes. DeepSeek-V4.1 joins the gate PCC sweep, and the
interleaved TP=1 case holds its deployed shape on the tuned entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ndow rings (9.1, 9.2)

Prefill's only DSpark state is each DSpark layer's window KV ring. TtV41DSpark
reduces the tap-layer block input streams (mean over hc copies), runs main_proj
(FP8 activation QDQ, row-parallel over TP, fp32 all-reduce) and main_norm on the
tile-aligned rows around the last window positions, then per DSpark layer
wkv -> kv_norm -> RoPE (theta 1e4, no YaRN, fp32) -> FP8 QDQ into ring slots
pos % window. All steps are row-local, so seeding per chunk gives the
single-shot ring (graph rule 6) with the ring as the only carry; padding rows
write nothing (rule 8).

Test vs the oracle (synthetic weights; real mtp shards are not downloaded):
prompts shorter/longer than the window and a chunked prompt whose last window
spans two chunks, small and real dims.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…acceptance (F1)

TtV41Moe builds the in-tree TtMoe for V4.1: 384 experts, sqrtsoftplus
DEVICE_FP32 gate, clamped SwiGLU, bfp8 or bfp4 routed experts (D-B).
Two numerics fixes came out of real-weight acceptance:
- The checkpoint's gate correction biases are about 10.76 +- 0.04, below
  the bf16 step at that magnitude (the device gate stores the bias in
  bf16), so top-6 recall fell to 0.90. The bias only selects experts,
  so the wrapper subtracts its mean; recall is back to 0.996.
- D-I: the reference FP8-QDQs every expert GEMM input but not the gate's.
  TtMoe.forward takes an optional expert_x (no-op when omitted) so the
  gate routes on x while dispatch and the shared expert read fp8_qdq(x).
  Shared expert PCC 0.9986 -> 0.9994 (bar 0.999).

test_moe_v41 runs the §6 oracle's layer-2 MoE input (5120 tokens) on 2x4
and 4x2, synthetic and checkpoint weights, bfp8 and bfp4, against the
reference Gate/Expert parts (cached by moe_reference, torch only). All
eight pass the G1 bars; test_dsv4_moe 4x2 rows reproduce their recorded
PCCs with the TtMoe change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TtV41Embedding reuses TtParallelEmbedding (bf16 table, hidden sharded over
TP), whose output already has the block layout, so no collective is needed;
the lookup is exact. TtV41Head narrows the collapsed final hidden to the
tile holding the requested row (M8: prefill needs one position), applies
the distributed final RMSNorm (eps 1e-20) and TtLMHead column-parallel
over the vocabulary with bf16 weights, fp32 activations and HiFi4/fp32
accumulation, returning fp32 logits like the reference ParallelHead.

test_head_v41 (2x4, 4x2; synthetic and checkpoint embed/norm/head):
embedding torch.equal to the reference; head logits PCC >= 0.99999 and
identical top-1 on 10 rows spread over SP chips and tile offsets,
including the oracle's own last-position logits; bit-identical repeats.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…F6.4)

TtV41Attention now runs the whole compressed-attention path on device over
V41PrefillState: KV sources run compressor, index keys and the compressed
KV write; index sources publish their top-k (the candidate source also its
candidate blocks) and consumers reuse it; index rows are compacted with a
top-k over validity and fed to sparse_sdpa on a head->sequence shard. FP8
activation QDQ precedes every quantized GEMM (dev-spec D-I). The state keeps
the per-chunk selection and the ratio-2 carry; blocks take (state, length)
and use TtV41Moe with an optional MoE weight cache. The host-fallback
prototype and its tests are removed.

Tests: tests/v41/test_block_v41.py runs layers 2,3,20,21,24 teacher-forced
with device-produced state, single chunk and two chunks, at small dims
(tests/v41/small_config.py, ~2 min per run) and real dims. Real weights,
real dims: blocks 0.992-0.9987, caches >= 0.9987, deterministic. Small
dims: layers 3-24 >= 0.9977 single and chunked; layer 2 (first, small
residual, synthetic weights) 0.982 -- reference-reproduced sensitivity, open.
Synthetic embeddings now use the checkpoint's RMS (0.12).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lock ladder

Synthetic norm weights and projection scales follow the released
checkpoint (norm means by type, projection std as a multiple of
fan_in^-0.5; measured on layers 0, 2, 20; user decision 2026-09-29): with
unit norms the MoE output swamped a small residual and first blocks of a
synthetic stack amplified tiny errors far beyond the real model. The
oracle result key excludes testing.py for checkpoint-weight specs, so
synthetic-init changes no longer invalidate real results. The small config
uses 128 experts (32 hung the in-tree routing setup) and a real-like
selected/visible ratio; the block test adds the SWA-only schedule.

Small dims on LoudBox 2x4 (3m17s for 4 runs): all six block types in the
schedules >= 0.9939, caches >= 0.9986, two chunks == one chunk,
deterministic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…shorter than a group

Index rows are padded to a multiple of 32 so sparse_sdpa always gets a
valid k chunk (a 3-token prompt produced 144 rows). The block test covers a
500-token prompt as 256 + 244 (padded tail, graph.md rule 8) and a 3-token
prompt (empty or single compressed row, empty top-k) for the sharing stack and
the SWA-only layer. Small dims: all pass, blocks >= 0.9943.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e (F5)

Engram (layers 1, 14) adds an n-gram table lookup into the 4 residual streams before
the block. Per dev-spec D-D the hash and the table lookup run on the host (token-only,
prefetchable): V41EngramHash reproduces the reference NgramHashState with the last 3
compressed ids carried across chunks (graph.md rule 5; padded tail never hashed, rule 8)
and V41EngramTable dequantizes the FP8/E8M0 rows as ParallelEngramEmbedding does.
TtV41Engram runs wkv (FP8 activation QDQ, columns permuted so each TP chip holds its
hidden slice of the 4 keys and the value) and the per-copy normalized-dot gate on the
packed fp32 streams; the per-copy sums are TP partials, all-gathered and added in chip
order so every chip computes the same gate. HiFi4 + fp32 accumulation on wkv: HiFi2
gives update PCC 0.99985 vs 0.99999 (bfp8 vs bf16 weights barely matter).

Tests vs the reference (cached CPU results in engram_reference.py): hash ids and
lookup bit-exact incl. chunked history and an image span across a chunk boundary;
device update >= 0.999 per copy, image rows unchanged, repeat bit-identical, at small
dims and at the production chunk 5120. Real Engram weights/tables (shards 47/48) are
not downloaded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TtV41Transformer runs a request chunk by chunk: embedding, 4-stream expansion,
one-hot pre-mix, the layers in order over V41PrefillState (DSpark taps and
per-chunk ring seeding when configured), and after the last chunk the final
collapse, norm and LM head on the last real token. It owns the MoE device
weight cache (per-layer completion marker; host MoE weights are skipped when
cached). Engram layers are rejected until F5 is wired. The block test keys
its weight cache by weight identity.

Small dims on LoudBox 2x4 (layers 0,2,3,20,21,24; 59 s for 3 cases): last-token
logits PCC 0.986 / 0.993 / 0.996 for one chunk, two chunks and a padded prompt,
top-1 match, bit-identical repeats.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Engram layers hash each chunk's valid tokens (carrying the n-gram history
across chunks), prepare their rows, and update the streams before their
block. The transformer test adds an Engram schedule (layers 0,1,2,3) and
checks top-1 only when the reference's own margin exceeds the device's logit
error (synthetic heads give near ties; G1 gates stacks on PCC). Small dims,
6 cases in 1.5 min: logits PCC 0.986-0.998, deterministic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Synthetic production-shape index-source blocks are gated at the real-weight
bar (0.98; user decision 2026-09-29): FP4-quantized top-k selection
disagrees intrinsically between bf16-accurate implementations (reference
self-recall 0.993 at bf16-level input perturbation) and synthetic attention
is ~7x more sensitive to those flips than real. Production shape on LoudBox
2x4, layers 2,3,20,21,24, one and two chunks: synthetic 0.9858-0.9993, real
0.9928-0.9995, caches >= 0.9988, deterministic.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
F6.1b asks for end-to-end cache contents >= 0.998 for every cache the prefill writes. The block test
checked compressed KV and index-K but not the window KV; it now compares each layer's window carry
(the last 128 positions' post-RoPE FP8-QDQ rows, right-aligned) with the oracle's window KV.
Small dims 12/12 (>= 0.9992), production synthetic + real (>= 0.9989).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remaining F5 acceptance: the table lookup on device and the host-vs-device decision.
Both table placements now dequantize on device from the stored bytes, chosen at
construction by the table object passed to TtV41Engram:

- host table (V41EngramTable): rows packed once (pack_rows: e4m3 bytes and E8M0
  scales rearranged into uint16 containers, 320 B/row); per chunk one host row
  gather, each chip gets 1/8 of the lookups (1 B/value instead of TP-replicated
  bf16), decodes them and all-gathers over TP.
- device table (TtV41EngramTable): packed rows row-sharded over the 8 chips plus a
  zero row per shard; only per-chip local row ids go up; every chip gathers
  (ttnn.embedding on the bf16-carried containers), the byte values (0..255, exact
  in bf16) are reduce-scattered over SP then TP, decoded and all-gathered over TP.
  Summing 16-bit containers in the reduction lost the low byte, hence bytes.

The device decode is exact vs the reference dequant as values (a table of every
finite e4m3 code, scales 2^-30..2^30); the device writes e4m3 -0 as +0.
Measured at a 5120-token chunk (synced wall): lookup host 7 ms + device 12 ms with
the device table, host 35 ms + device 7 ms with the packed host table, host 73-98
ms with the previous bf16 upload. The device table needs 15.4 GB/chip per table on
2x4, so LoudBox keeps the host table; see the bead for the decision record.

oracle.synthetic_engram_rows draws rows in fp32 regardless of the caller's default
dtype (under set_dtype(bf16) it produced different rows); fp32-default results are
unchanged, so cached oracles stay valid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DeepSeek-V4.1 attends over one 512-dim KV row with RoPE inside it; its stage-2 cache
format (epic KV FORMAT) is scaled FP8 over all 512 dims: [512 FP8 | 4 FP32 scales],
528 B per row. The shared ops assumed the DSA 512+64 geometry:

- pack_scaled_fp8_kv_cache: rope input is optional; latent / scale / RoPE widths come
  from the inputs (latent a multiple of 128) using the sparse_sdpa_common.hpp row
  geometry. The program hash keys on RoPE presence and shape; the RoPE copy compiles
  out when absent (placeholder accessor args).
- sparse_sdpa SCALED_FP8 with K_DIM == v_dim: no RoPE CBs, the RoPE tilize and the
  RoPE partial QK matmul are compiled out; the RoPE-offset alignment check applies
  only when a RoPE tail exists. Changes sit inside the scaled_kv branch; BF16 and
  FP8_E4M3 paths are unchanged.
- MlaKvCacheGeometry accepts rope_dim 0.

Tests: new rope-less pack cases (512 and one-scale 128 rows) and sparse_sdpa
scaled-FP8-without-RoPE cases (512 at H 32/64, 128). Consumers re-run: post-commit
test_sparse_sdpa.py (50/50), nightly pack test (6/6), sparse KV cache contract
(15/15), sparse_mla SCALED_FP8 chunked/pad-overflow/accuracy on 2x4 (5/5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…me format (F6.2)

Stage 2 of the epic KV FORMAT: the compressed layers' KV tensors (window slot, chunk
window rows and compressed rows share one sparse_sdpa call, so one format) are stored
as [512 FP8 e4m3 | 4 FP32 power-of-two scales] per row, quantized per 128 dims (RoPE
included) from the unrounded post-RoPE KV, never from the FP4/FP8-QDQ values.

- V41PrefillState(kv_format=MlaKvCacheFormat.BF16_RM | SCALED_FP8), default BF16_RM
  (unchanged behavior). The state encodes rows it writes (per_token_cast_to_fp8 +
  rope-less pack) and decodes them for host checks (to_host). Ratio-0 (SWA-only)
  layers stay BF16 in both stages: they hold the reference FP8 block-32 window values
  exactly and share no tensor with compressed rows. Index-K stays BF16.
- TtV41Attention applies the reference QDQ only at the BF16 stage and passes the
  layer's format to sparse_sdpa.
- test_block_v41 gains kv_format {bf16, scaled_fp8}. At scaled_fp8 compressed KV is
  compared with the reference's compressed KV before its FP4 QDQ (derived from the
  reference compressor on the recorded attention input; the test asserts its FP4 QDQ
  reproduces the oracle rows bit for bit). The PCC against the FP4-QDQ oracle rows is
  reported only: it is ~0.995 by construction (FP4 vs unrounded), below 0.998.

Also includes the coordinator's DSpark ring slot (dspark_rings) in cache.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TtV41Moe sized the dispatch buffer with capacity factor 2. Repetitive real text concentrates routing
on one chip's experts; the overflowing tokens were dropped silently and in arrival order, so a small
Engram stack on a real-text prompt was neither correct nor bit-identical across repeated prefills.
Top-k (6) is the only factor that bounds the worst case (a token can land all its experts on one
chip); in-tree DeepSeek-V3 prefill uses its top-k (8) the same way. With it the repeats are
bit-identical again (diagnosis: evidence/F9-drift-diagnosis/diag_cap.log).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eference noise floor (F9)

Free-running V4.1 stacks are chaotic: top-k selection and MoE routing flip under tiny input error
(real layer 20's MoE keeps the same experts on 77% of rows at input PCC 0.9987; the reference's
own last-token logits move to PCC 0.968 under 0.1% embedding noise), so an absolute bar on one
token's logits measured the flips, not the device. User decisions 2026-09-29:
- prompts are real text (oracle.text_tokens: A Tale of Two Cities, the tt_transformers accuracy
  book); a prefill is teacher-forced by construction, so the last 256 positions are scored
  (repo convention, e.g. DeepSeek-V3 teacher-forced top-1/top-5);
- bar = the reference's worst self-agreement over 5 seeds (oracle.tail_logits with noise) under
  4.5% noise on every attention and MoE output (the error a component bar of 0.999 allows) plus
  0.4% on every attention input (where the reference's layer-20 attention matches the device's),
  minus sampling margins; top-1 agreement, top-5 recall and logits PCC;
- DSpark rings from free-running streams are reported, not gated (the 0.998 ring bar applies with
  teacher-forced taps, gated in test_dspark).
Device: TtV41Transformer.prefill(tokens, logit_positions) returns the logits of the last N prompt
positions (TtV41Head.rows_to_host, one projection per 32-row tile) and keeps the DSpark rings in
V41PrefillState. Tests: production-shape stack (layers 0 2 3 20 21 24, S=2048, synthetic + real),
small DSpark schedule (20, 36-39), stage timing logs, identity-keyed device weight caches at small
dims. Small 9/9 + oracle CPU test pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…orted meshes rejected (11.1)

Every V4.1 module already derives SP/TP from mesh_device.shape, but nothing stated
which meshes the layer can run on: an unsupported mesh failed deep in an op (or,
for MoE, silently floored experts // chips). tt/v41/layout.py (V41MeshLayout) now
owns the SP=0 / TP=1 axis convention and the layout requirements:
- model (checked by TtV41Block): hidden/tp whole tiles, heads % tp, heads % 32
  (sparse_sdpa after the head->sequence all-to-all), O_GROUPS % tp,
  routed experts % chips;
- chunk (checked by V41PrefillState): chunk % (32*sp*tp), whole-tile query rows
  per chip for the head->sequence attention and the indexer query split (the SP
  rule 2*32*sp stays with V41CacheGeometry).
Both accept LoudBox 2x4 / 4x2 and Galaxy 8x4 / 4x8 at real and small dims; 2x4
behavior is unchanged (all existing chunks satisfy the new rule). The perf model
no longer defaults to the LoudBox 2x4 layout (every caller passes one).

Tests: tests/v41/test_v41_layout.py (CPU, parametrized over the four layouts plus
rejection cases); 2x4 block + transformer small cases and perf-model tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he prototype oracle (§6)

The vision test built its own expected values on every run (up to ~2 min of CPU per
1920x1080 case). The oracle package now owns them: vision_oracle runs the reference
encoder (vision.ViT -> vision.Aligner, as Transformer.encode_image) on synthetic
(per-tensor seeded) or checkpoint (shard 1, as stored) weights and caches the result
keyed by args, weights source, patch digest, grid, vision.py digest, torch version and
PACKAGE_VERSION. Test inputs (synthetic PNGs through image_processor, direct patch grids)
come from the package too. Existing oracle results and cache keys are unchanged.

The prototype oracle's schedule is real_spec((2, 3, 20, 21, 24), 2048,
candidate_topk_blocks=96); test_indexer now uses it (V4.1 layer ids). device_weights /
dequant are a device-layout helper, not an oracle: moved to tests/v41/reference_weights.py,
with include_moe=False to skip the routed-expert FP4 dequantization for callers whose
device MoE tensors are cached. prototype_oracle.py keeps only a device_weights re-export
until test_block_v41 / test_transformer_v41 switch imports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kbot added 21 commits November 12, 2026 22:05
Several whole prompts concatenated in one generic chunk (contiguous from
row 0, ratio-aligned lengths, the chunk at start 0), each isolated from
the others by the generic chunk inputs it already takes plus three
host-side extensions: V41ChunkTables fills RoPE rows per segment (local
positions), keeps a resident window-row table per layout (-1 where a
query would read the previous prompt), and adds a per-query lower bound
to the indexer masks (the first compressed row of the query's segment,
one lt and one logical_or per mask) so no selected row lies before the
segment; the attention compacts every row of the chip (a segment can
start anywhere on it); the state writes each segment's window carry into
its own buffer (segment_carry) beside the chunk's; each prompt is
Engram- hashed from an empty history; the head scores each segment's
last row. TtV41Transformer.prefill_packed runs one such chunk eagerly
and returns one logits row per prompt. With the flag off (the default)
no graph changes: the extra mask ops and the layout table exist only
while a packed layout is being run. tests/v41/test_v41_pack.py compares
each segment, bit for bit, with the unpacked prefill of the same prompt
on the same geometry (logits, cache rows in segment coordinates,
carries, selections), mutates the earlier neighbour, runs two negative
controls (a cross-segment window table, the lower bound dropped), checks
the flag-off state, and times the single-prompt chunk and each pack
(eager, and captured and replayed with V41_PACK_TRACE=1).

Action-Id: ds41/optimize/233
The packed-chunk test reads every chip copy as [chips, 1, rows, W] (a
stacked 5-D readback sliced the wrong axis for the row-major caches),
records per key the differing rows, compares each selection as a set
(sorted ids) beside its score order, probes each block's norm, attention
and FFN outputs and, one level down, the attention's projections, TP
all-reduces and RoPE (V41_PACK_PROBE=1|2), and packs prompt B behind a
first segment of V41_PACK_OFFSETS lengths. Its assertions are the
isolation contract (the selection ranges, the flag off, segment 0, every
segment under its neighbour's mutation, determinism, the negative
controls); a later segment against its single run at row 0 is reported
and asserted only under V41_PACK_STRICT=1, since it differs by the TP
all-reduce's row-dependent summation order and the top-k's tie order.
V41ChunkTables records the flag (_pack) so a state without a ratio > 0
layer can run a pack too.

Action-Id: ds41/optimize/233
Per-segment joins behind TT_V41_PACK with TT_V41_PACK_LAYOUTS:
V41DecodePool.join_inputs / join_ops / join take segment=(offset,
length, g) (the compressed rows and index keys copied from offset /
ratio, the index covering length / ratio rows, the segment's own window
carries, no compressor carry), TtV41Decoder.join / capture_join carry it
(the user position is the segment's length: the masks' and the c1
live-user index's bound; the capture key adds the offset and carry
index), V41PrefillState allocates pack_segments carries at construction
and the layouts' tables on request, and the generator builds one packed
state at the first bucket, runs the FIFO prefix of a call's whole
prompts that forms a listed layout as one packed chunk (eager; its rows
pending on their segments, joined at their first decode step or when the
next pack needs the state), prebuilds the layouts' totals' padding
configs and runs one eager pack with its segment joins in the compile
pass. tests/v41/test_v41_pack_join.py is the decode-token gate.

Action-Id: ds41/optimize/233
V41PackTrace: the packed chunk of one layout captured once on the packed
state after the eager warm-up runs and replayed for every pack of that
layout (the token ids, Engram inputs and the layout's fills copied
before the replay; the k scored rows read after it; the replay's device
time in last["wait"]). The generator's trace pass captures every listed
layout and then the joins of its segments into every slot at the
segments' width, after the prefill states' joins and generic captures
and before the decode warm-up's trace pass; TT_V41_PACK_TRACE=0 keeps
the packs eager. Every persistent tensor of the packed path exists
before the decode captures (the compile pass of the previous commit).

Action-Id: ds41/optimize/233
The generator built the MoE padding configs of the chunk, the warm-up
prompt, the listed trace lengths and the packed layouts' totals before
the decode capture, but not of a layout's segment lengths: a packable
prompt that runs alone (the join test's references, which list no trace
length; a queue of one in serving) built its config lazily, after the
decode capture, where the decode replays overwrite it. In the join test
the k=2 references of the same prompt drifted from their k=4 run (PCC
0.97 at 40 layers, another argmax at 7), and the traced 40-layer run
hung in the first eager chunk after the c1 replays. The compile pass now
runs one eager pack of every listed layout, not the largest alone, so no
layout's first run follows a capture. The test asserts the references
stable across the decode replays and segment 0 of every pack equal to
the kmax pack's.

Action-Id: ds41/optimize/233
The join test read the candidate source's block pages whole, so the last
partial page carried the slot's previous occupant's rows past the
prompt's length: fresh slots agreed in the first phase, reused ones
differed in the next (446 of 1,024 elements of page 128 of 129), as the
single path's slots would. The pages are now decoded to rows and cut at
the prompt's rows. The c1 token comparison is reported rather than
asserted: the reference and the packed slot step alone at different pool
steps, their ring rows at different physical slots, so the window's
reduction order differs by rotation and a near-tie of the 7-layer model
flips at step 3; the same-step comparison is the bit-exact gate. And
V41_PJ_ROWS=0 skips the pool-row download: the test reads every pool
tensor of every chip to the host, hundreds of GB at the served 1M
context with 32 slots, where the OOM killer ended it.

Action-Id: ds41/optimize/233
prefill_packed filled the pack's position tables with its own unbatched
copies after _stage; the current _stage fills a generic state's tables
itself (TT_V41_PREFILL_HOST_BATCH=1 puts them in the one dispatch
round), and it takes the pack's layout since the rebase, so the packed
path uses it too.

Action-Id: ds41/optimize/233
The packed prefill state was built on the full-chunk model, which is
10,240 rows now: a pack of four 1,028-token prompts would pay that chunk
(~0.65 s) where measurements showed 0.29 s on the 5,120 one, no better
than the four single prefills it replaces (0.18 s each on the 2,048
geometry). TT_V41_PACK_CHUNK names one of TT_V41_SHORT_CHUNK's
geometries to build the state on; the layouts are checked against that
chunk, and the warm-up's eager pack and captures run on that view.
Unset, the full-chunk model as before.

Action-Id: ds41/optimize/233
A packed chunk (TT_V41_PACK) runs several whole prompts side by side,
each at its own positions, but the chunk's DSpark seeding wrote the
chunk's last window positions into the state's one ring set as if it
were one prompt, so a packed segment could not join a decode slot with
the drafter on (TT_V41_SPEC): the generator asserted on it.

The packed state now holds one ring set per segment
(V41PrefillState.segment_rings[g], allocated with the segment carries by
pack_segments=), the chunk tables keep a seed-row RoPE pair per segment
(window + 64 rows: a segment's offset is ratio-aligned, not
tile-aligned) filled per layout with the pack's other host inputs
(prepare_pack_host), and TtV41DSpark.seed_packed projects the taps
gathered once at each segment's tile-aligned rows and writes that
segment's last min(length, window) positions into its own rings at slots
p % window: row-local throughout, so a segment's rings equal the single
prefill's of the same prompt. seed_rows takes the chunk-row offset;
seed() is unchanged in effect. prefill_packed slices the pairs inside
the forward, so V41PackTrace captures the slice and a replay reads the
host-filled buffers. The generator's join loads a segment's rings into
the slot's drafter rings at the segment's length (_load_rings, shared
with the single path), and the warm-up's pack joins do the same so the
warm-up rounds draft from coherent rings. test_v41_pack_join.py compares
each slot's drafter ring rows of the prompt's last 128 positions after
the join (bit-equal for segment 0) and records the device rounds each
slot took under the speculative round, which must agree between the
single and the packed prefill of one prompt.

Action-Id: ds41/optimize/234
TtV41VerifyDecoder.join overrides TtV41BatchDecoder.join without the
segment argument the packed chunk's join takes (TT_V41_PACK), so the
first segment join with the drafter on (the warm-up's) raised a
TypeError. The override takes and forwards it; a packed segment is
ratio-aligned and brings no compressor carry, so the multi-slot carry
row below it is untouched.

Action-Id: ds41/optimize/236
Two defects of the pack-join test under the served stack. The pool's
window rings are 160 position-based slots with the verify decoder
(TT_V41_SPEC), and a join of a 1,028-token prompt writes its last 128
positions at q % 160, leaving 32 rows to an earlier occupant; the test
read the slot's first 128 ring rows blindly, so a replayed pack's slot
and an eager pack's slot differed in the rows a previous decode wrote
(68..80 of 128) while every written row was equal. The rows are now the
prompt's last 128 positions at their slots (the whole ring where the
slot is the pool step's). And the single reference's geometry is judged
against the pack state's chunk (TT_V41_PACK_CHUNK's view), not the
model's: on the served geometry the reference prefills eagerly on the
10,240 chunk while the pack sits on the 5,120 view, so segment 0 is
reported there and asserted where the two share a chunk.

Action-Id: ds41/optimize/238
tests/v41/test_v41_round_profile.py drives the generator as the plugin's
ragged contract does (after test_v41_spec_generator.py): the two-phase
warm-up, V41_RP_LIVE requests prefilled from distinct prompts of
V41_RP_PROMPT tokens, then warm and measured rounds, one decode_forward
a round. V41_RP_PROFILE=1 reads the device profiler after the warm-up,
the prefills and every round, so each measured round is one replay
session of its capture in the C++ report. V41_RP_OPMAP records, while
the captures are made, which model function issued which device
operation ids, so a replayed op is attributed by its id. Test only:
nothing the server runs is changed.

Action-Id: ds41/optimize/253
V41_RP_DRAIN_SKIP names the reads to skip (warm-up, prefills, ...). With
the raw device log written, every read dumps what the buffers hold; a
phase that is not read only overflows them and its markers are dropped,
which bounds the log to the rounds that are measured. Test only.

Action-Id: ds41/optimize/256
… one chain

The verify step selected each live tile row's top-k among its candidate
blocks with its own chain of 25 programs (select_user_in_blocks: the
row's query repeated over 32 rows, candidate_scores, a top-k of a tile
with one real row, the id arithmetic), 4 chains a consumer source in the
one-slot round and 16 in the 32-slot one, every program at the launch
floor whatever the row.

TtV41Indexer.select_rows_in_blocks runs one chain for all of them:
candidate_scores takes the live rows as (tile row, block base) pairs and
ids of s rows per live row, reads each row's query, head weights and pin
from its own tile row and its blocks from its own user's pages (QSPLIT
in the reader and the writer; the per-row launches are unchanged), so
one launch scores every live row's blocks on up to all cores; the scores
are a row per live row again, one top-k and one id chain work on those
rows. TtV41VerifyDecoder._index takes it under V41_VR_MERGE_BLOCKS=rows
(default row: the per-row chain) and hands it the candidate source's
stacked blocks without slicing them apart. No constant is allocated.

test_v41_select_rows_in_blocks: the merged selection against the per-row
one, bit-equal on every chip, in the one-slot and the 32-slot layouts.

Action-Id: ds41/optimize/263
The round's record carried every row's emitted count; with the tokens
two sessions on the same prompts are compared token for token (a flag's
exactness on 32 distinct prompts). Test only.

Action-Id: ds41/optimize/263
The verify step scored each densely scored live tile row with its own
range_scores launch (32 cores, read-bound, most of a launch at a short
context the fixed -inf fill of its row of the pool's width) and then
concatenated the rows' split layouts for the merged selection: 4
launches a source in the one-slot round, 16 in the 32-slot one, and a
copy of every score.

range_scores(live=[(user, row), ..]) scores them in one launch that
writes the stacked layouts itself: a unit is one split row of one live
row (key user, split row, query row, output row), the units go round the
first min(L * rows, grid) cores, and the three kernels work a core's
units one after another (the reader keeps every unit's bound before it
hands the pin page's buffer over; the writer's two chunk buffers
alternate across units). The single launch is one unit a core, as
before. TtV41VerifyDecoder._index takes it under
V41_VR_MERGE_SCORES=rows where the merged selection runs (default row: a
launch a row and the concat).

test_v41_range_scores_rows: the one launch against a launch a row,
bit-equal on every chip, 4 and 16 live rows, one and 32 split rows. The
consumers' test shares its inputs through helpers.

Action-Id: ds41/optimize/265
For a sentinel block id the reader read the slot's block 0 (finite
values the writer masks to -inf). On a chip whose rows are all idle, the
28 padding chips of a one-slot step, every id is the sentinel and every
core reads that one page K times: 86.6 us a launch there against 20 on
the live rank's chips, whose ids are spread over the pool.

V41_CANDSCORE_SENTINEL=skip issues no read for a sentinel id: the
block's scores are whatever the buffer held and the writer masks them as
before (a block's scores depend on its own rows only). Default read.

test_v41_candidate_scores_skip_sentinel: the selections bit-equal
between the two, rows with sentinel tails and idle rows; a launch over
idle rows timed both ways.

Action-Id: ds41/optimize/266
The 32-slot verify gate hung the device with range_scores(live=) on: 16
live rows a rank on a ratio-4 source of the 1,054,720-row pool, 263,712
columns in 32 split rows of 33 chunk slots, the 300-token prompt in
every slot. test_v41_range_scores_rows checks the same 16 rows on a pool
of 20,480 rows, three chunk slots a split row.

test_v41_range_scores_rows_wide runs the one launch against a launch a
row at the production width, 4 and 16 live rows, with the gate's
lockstep positions (one non-empty split row a live row) and with the
spread ones; the keys past row 20,480 are zero.

Action-Id: ds41/optimize/268
With 32 split rows, 4 live rows (128 units) and 16 (512 units) both
launch on the grid's first 120 cores with the same compile-time
arguments, so they share a program-cache entry. On a hit the generic op
re-applies each core's runtime arguments only up to the length the first
call gave that core. The 32-slot round's verify step after its plain
step makes exactly that sequence, and hung the device.

test_v41_range_scores_rows_cache calls the one launch with the 4 rows of
the one-slot layout, then the 16 of the 32-slot layout, then the 4
again, each against a launch a row.

Action-Id: ds41/optimize/270
…ngth

The generic op's program-cache key holds each kernel's cores and
compile-time arguments but not each core's runtime-argument count, and a
cache hit re-applies a core's arguments only up to the length the
entry's first call gave it. range_scores(live=) gave a core 3 words a
unit: 4 live rows (1 or 2 units a core on 120 cores) then 16 (4 or 5)
reused the first call's program with the later units cut off, the
reader, compute and writer disagreed on each unit's chunks, and the
32-slot verify step hung the device.

Every core's reader, writer and compute arguments are now padded to
RANGE_MAX_UNITS units, so the length is one for all calls and a hit
carries every unit. The kernels are unchanged: they read n_units units.

Action-Id: ds41/optimize/270
The three flags pass every exactness gate of the verify round (one slot
at K = 1, 2, 3, 32 slots, the ring's wrap, the 1M pool's end) and pay
their estimates at 32K: of verify device a round, the one-slot round
68.7 -> 65.0 ms and the 32-slot round 105.1 -> 89.7, the emitted tokens
equal.

V41_VR_MERGE_BLOCKS and V41_VR_MERGE_SCORES now default to rows and
V41_CANDSCORE_SENTINEL to skip, so the served configuration takes them
without a catalog line; row, row and read keep the per-row paths.

Action-Id: ds41/optimize/274
for u in rng.sample(range(4), blocks):
p = rng.randint(0, pmax)
a = rng.randint(0, 4)
taps = [(rng.randint(0, db.ROWS - 1), p - a + j) for j in range(a + 1)]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗Cycode: SAST violation: 'Usage of weak pseudo-random number generator (Python)'.

Risk Score: 64 (MEDIUM)
Severity: High

Description

The random module in Python uses the Mersenne Twister algorithm, a pseudorandom number generator (PRNG) whose output is deterministic and reproducible. Given enough observed values, an attacker can reconstruct the internal state and predict future and past outputs. Because code evolves and values generated by random may flow into security-sensitive contexts over time (e.g., identifiers, tokens, session keys, or access-control decisions), using a predictable source of randomness anywhere in the codebase increases the attack surface.

Cycode Remediation Guideline

✅ Do


  • Do use the secrets module for generating random values. It provides cryptographically secure randomness, making it a safe default regardless of how the values are used downstream.
  • Do use os.urandom() or random.SystemRandom() when you need a secure source of randomness compatible with the random API.

❌ Don't


  • Do not use random for generating values that may be exposed externally or used in security-relevant logic, including identifiers, tokens, nonces, passwords, or access-control decisions.
  • Do not treat the usage context of random values as fixed — over time, refactoring and feature growth can expose previously internal values to security-sensitive flows.

📋 References


a = rng.randint(0, 4)
taps = [(rng.randint(0, db.ROWS - 1), p - a + j) for j in range(a + 1)]
taps = [(t, q) for t, q in taps if 0 <= q <= p] if rng.random() < 0.9 else []
es.append({"user": u, "token": rng.randint(0, 100000), "position": p, "taps": taps})

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗Cycode: SAST violation: 'Usage of weak pseudo-random number generator (Python)'.

Risk Score: 64 (MEDIUM)
Severity: High

Description

The random module in Python uses the Mersenne Twister algorithm, a pseudorandom number generator (PRNG) whose output is deterministic and reproducible. Given enough observed values, an attacker can reconstruct the internal state and predict future and past outputs. Because code evolves and values generated by random may flow into security-sensitive contexts over time (e.g., identifiers, tokens, session keys, or access-control decisions), using a predictable source of randomness anywhere in the codebase increases the attack surface.

Cycode Remediation Guideline

✅ Do


  • Do use the secrets module for generating random values. It provides cryptographically secure randomness, making it a safe default regardless of how the values are used downstream.
  • Do use os.urandom() or random.SystemRandom() when you need a secure source of randomness compatible with the random API.

❌ Don't


  • Do not use random for generating values that may be exposed externally or used in security-relevant logic, including identifiers, tokens, nonces, passwords, or access-control decisions.
  • Do not treat the usage context of random values as fixed — over time, refactoring and feature growth can expose previously internal values to security-sensitive flows.

📋 References


p = rng.randint(0, pmax)
a = rng.randint(0, 4)
taps = [(rng.randint(0, db.ROWS - 1), p - a + j) for j in range(a + 1)]
taps = [(t, q) for t, q in taps if 0 <= q <= p] if rng.random() < 0.9 else []

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗Cycode: SAST violation: 'Usage of weak pseudo-random number generator (Python)'.

Risk Score: 64 (MEDIUM)
Severity: High

Description

The random module in Python uses the Mersenne Twister algorithm, a pseudorandom number generator (PRNG) whose output is deterministic and reproducible. Given enough observed values, an attacker can reconstruct the internal state and predict future and past outputs. Because code evolves and values generated by random may flow into security-sensitive contexts over time (e.g., identifiers, tokens, session keys, or access-control decisions), using a predictable source of randomness anywhere in the codebase increases the attack surface.

Cycode Remediation Guideline

✅ Do


  • Do use the secrets module for generating random values. It provides cryptographically secure randomness, making it a safe default regardless of how the values are used downstream.
  • Do use os.urandom() or random.SystemRandom() when you need a secure source of randomness compatible with the random API.

❌ Don't


  • Do not use random for generating values that may be exposed externally or used in security-relevant logic, including identifiers, tokens, nonces, passwords, or access-control decisions.
  • Do not treat the usage context of random values as fixed — over time, refactoring and feature growth can expose previously internal values to security-sensitive flows.

📋 References


es = []
for u in rng.sample(range(4), blocks):
p = rng.randint(0, pmax)
a = rng.randint(0, 4)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗Cycode: SAST violation: 'Usage of weak pseudo-random number generator (Python)'.

Risk Score: 64 (MEDIUM)
Severity: High

Description

The random module in Python uses the Mersenne Twister algorithm, a pseudorandom number generator (PRNG) whose output is deterministic and reproducible. Given enough observed values, an attacker can reconstruct the internal state and predict future and past outputs. Because code evolves and values generated by random may flow into security-sensitive contexts over time (e.g., identifiers, tokens, session keys, or access-control decisions), using a predictable source of randomness anywhere in the codebase increases the attack surface.

Cycode Remediation Guideline

✅ Do


  • Do use the secrets module for generating random values. It provides cryptographically secure randomness, making it a safe default regardless of how the values are used downstream.
  • Do use os.urandom() or random.SystemRandom() when you need a secure source of randomness compatible with the random API.

❌ Don't


  • Do not use random for generating values that may be exposed externally or used in security-relevant logic, including identifiers, tokens, nonces, passwords, or access-control decisions.
  • Do not treat the usage context of random values as fixed — over time, refactoring and feature growth can expose previously internal values to security-sensitive flows.

📋 References


entries = []
for _r in range(8):
es = []
for u in rng.sample(range(4), blocks):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗Cycode: SAST violation: 'Usage of weak pseudo-random number generator (Python)'.

Risk Score: 64 (MEDIUM)
Severity: High

Description

The random module in Python uses the Mersenne Twister algorithm, a pseudorandom number generator (PRNG) whose output is deterministic and reproducible. Given enough observed values, an attacker can reconstruct the internal state and predict future and past outputs. Because code evolves and values generated by random may flow into security-sensitive contexts over time (e.g., identifiers, tokens, session keys, or access-control decisions), using a predictable source of randomness anywhere in the codebase increases the attack surface.

Cycode Remediation Guideline

✅ Do


  • Do use the secrets module for generating random values. It provides cryptographically secure randomness, making it a safe default regardless of how the values are used downstream.
  • Do use os.urandom() or random.SystemRandom() when you need a secure source of randomness compatible with the random API.

❌ Don't


  • Do not use random for generating values that may be exposed externally or used in security-relevant logic, including identifiers, tokens, nonces, passwords, or access-control decisions.
  • Do not treat the usage context of random values as fixed — over time, refactoring and feature growth can expose previously internal values to security-sensitive flows.

📋 References


def _instrument(block, sink, mesh_device):
def wrap(owner, attr, name):
if getattr(owner, attr, None) is not None:
setattr(owner, attr, _Timed(getattr(owner, attr), name, sink, mesh_device))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗Cycode: SAST violation: 'Unsanitized external input in code generation (Python)'.

Risk Score: 74 (HIGH)
Severity: Critical

Description

Allowing external input (dynamic or user-controlled) to directly influence code generation or scripting functions without proper sanitization can lead to code injection vulnerabilities. This occurs when an attacker is able to insert malicious code into your application, which is then executed, potentially leading to unauthorized actions or data access.

Cycode Remediation Guideline

✅ Do


  • Do always validate or sanitize input to ensure it does not contain harmful code before using it in such contexts.

❌ Don't


  • Do not pass unsanitized external input to functions or methods that dynamically execute code.

📋 References


for _r in range(8):
es = []
for u in rng.sample(range(4), blocks):
p = rng.randint(0, pmax)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗Cycode: SAST violation: 'Usage of weak pseudo-random number generator (Python)'.

Risk Score: 64 (MEDIUM)
Severity: High

Description

The random module in Python uses the Mersenne Twister algorithm, a pseudorandom number generator (PRNG) whose output is deterministic and reproducible. Given enough observed values, an attacker can reconstruct the internal state and predict future and past outputs. Because code evolves and values generated by random may flow into security-sensitive contexts over time (e.g., identifiers, tokens, session keys, or access-control decisions), using a predictable source of randomness anywhere in the codebase increases the attack surface.

Cycode Remediation Guideline

✅ Do


  • Do use the secrets module for generating random values. It provides cryptographically secure randomness, making it a safe default regardless of how the values are used downstream.
  • Do use os.urandom() or random.SystemRandom() when you need a secure source of randomness compatible with the random API.

❌ Don't


  • Do not use random for generating values that may be exposed externally or used in security-relevant logic, including identifiers, tokens, nonces, passwords, or access-control decisions.
  • Do not treat the usage context of random values as fixed — over time, refactoring and feature growth can expose previously internal values to security-sensitive flows.

📋 References


This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants